audio: tensorflow: keyword detect - #11092

See thesofproject#11097

The two stft_process source files pulled in libc's <assert.h>,
which under a newlib-based Zephyr build (e.g. MTL with
COMMON_LIBC_MALLOC_ARENA_SIZE set) expands assert() to a
__assert_no_args() call whose implementation is not linked into
the SOF firmware image. Enabling COMP_STFT_PROCESS on such a
build therefore fails at link time with an "undefined reference
to __assert_no_args" error.

Every other SOF audio component that uses assert() includes
<rtos/panic.h> instead, which maps assert() to Zephyr's
__ASSERT_NO_MSG in firmware builds and to sof_panic() in the
posix testbench. Do the same in stft_process-generic.c and
stft_process-hifi3.c.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Wire the TFLM sources into the SOF firmware build for statically linked
firmware. This unblocks non-LLEXT targets where TFLM has to live in
the base image, using the Zephyr SDK GCC HiFi3 toolchain.

Also add the requantize helper in speech.cc that unpacks the IEEE 754
mantissa directly instead of calling frexpf, so the file pulls in no
libm symbols on the minimal-libc SOF build.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

This model was trained with scripts

- sof_tflm_generate_keyword_dataset.sh
  with parameters MAX_SAMPLES=1000, MAX_SPEAKERS=200, SLERP_WEIGHTS=0.0.

- sof_tflm_train_pipeline.sh
  with parameters WINDOW_HOP_STEP=5, N_SILENCE=1500, N_UNKNOWN=4000,
  EPOCHS=40.

below is information from model verification:

samples: 7693  overall accuracy: 0.9956

  class       support  precision   recall      f1
  -----------------------------------------------
  silence         225     0.9782   0.9956  0.9868
  unknown         736     0.9849   0.9728  0.9788
  banana         2546     0.9961   0.9961  0.9961
  strawberry     1748     0.9971   1.0000  0.9986
  orange         2438     0.9988   0.9988  0.9988

confusion matrix (rows=true, cols=pred):
                 silence     unknown      banana  strawberry      orange
  silence            224           0           1           0           0
  unknown              5         716           9           3           3
  banana               0          10        2536           0           0
  strawberry           0           0           0        1748           0
  orange               0           1           0           2        2435

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Introduce a TensorFlow Lite Micro-based keyword-detection component
that runs an int8-quantized micro-speech model on the mel features
from the MFCC component. The key-phrase buffer (KPB) sits ahead of
MFCC in the pipeline; on a positive keyword the TFLM component posts
a KPB trigger notification so the host can drain the buffered audio.

Also includes tensor byte-sizing fixes for the int8 input path,
per-hop mel feature requantization that strips the MFCC header and
avoids float math on the DSP, and stream-shutdown event logging so
the host sees a clean end-of-capture on trigger.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The on-device TFLM wake-word path (KPB -> SRC -> MFCC -> tflmcly)
needs a 40-bin/20 ms-hop mel spectrogram to match TFLM
micro_speech's front-end shape (FEATURE_SIZE=40,
FEATURE_STRIDE_MS=20, FEATURE_DURATION_MS=30). Two variants are
required: mel40.conf for the plain audio-buffer path used for
off-target sanity checking and mel40_compress.conf for the
compress PCM output actually consumed by the wake-word inference
component.

Add both export blocks to setup_mfcc.m so the blobs stay in
lock-step under any future front-end tuning change, and ship the
initial generated mel40.conf and mel40_compress.conf so the
Wake-on-Voice topology graphs that consume them have matching
blobs to reference.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

When the MFCC component runs in the DP (Data Processing) scheduling
domain, the DP thread is woken on ibs, which is roughly one FFT hop
of input. Sizing the processing capacity to only dev->frames means
the DP thread has to re-enter many times per LL tick to nibble
through one hop, adding unnecessary scheduling overhead.

Raise the max_frames argument passed to mfcc_setup() to at least
cd->config->frame_shift when the component is scheduled in the DP
domain, so a single DP invocation drains one whole hop.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Enable KPB, TFLM, MFCC, Gain and Volume on the cAVS 2.5 (TGL/ADL/RPL)
board defconfig and size its DP scheduler heap for the TFLM working
set. Register the DP scheduler in the cAVS platform init so the DP
domain pipelines that host MFCC and TFLM come up cleanly, and add
tflmcly.toml to the TGL rimage manifest so the signed image lists
the new modules.

Also make the tensorflow static CMake wiring portable across xtensa
targets: hardcoded /home/lrg zephyr-sdk paths, mcpu names and include
directories are replaced with SOC_TOOLCHAIN_NAME and ZEPHYR_SDK_INSTALL_DIR
expansions, the -mllvm text-section-literals flag is feature-detected,
and HiFi4-only kernels are gated on TENSORFLOW_HAVE_NNLIB_HIFI4.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Document the TFLM Wake-on-Voice architecture: pipeline graph,
module UUIDs, topology entry points, static vs LLEXT build
variants, MFCC config blob layout, KPB trigger flow, and the
on-device vs off-device verification loop.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add an end-to-end training and verification toolchain for TFLM wake-word
models under src/audio/tensorflow/tune:

  - Piper-TTS driven keyword dataset generator with per-file Gaussian
    gain jitter capped 3 dB below full scale so the augmented WAVs
    directly reflect the level distribution without clipping.
  - Batch feature extractor that runs the SOF testbench MFCC on the
    WAVs and writes raw mel40 features.
  - Silence and unknown-class preparation from Speech Commands v2 with
    the same level-jitter policy (silence exempt).
  - Python trainer that fits an int8-quantized DS-CNN on the mel
    features, emits the tflite/C-array pair, and auto-writes a
    sof_tflm_labels.h header so the runtime labels match the training
    set without hand-editing.
  - Off-device verifier that runs the tflite model against a directory
    of WAVs and reports per-class scores.
  - Streaming diagnostics plotter (sof_tflm_plot_mtrace.py) that parses
    SOF mtrace logs to visualize energy, noise floor, mel envelopes,
    quantized features, AGC gain tracking, and keyword detection events.

Refreshes the tensorflow install and training recipe and fixes int8
calibration to use the correct scale and zero-point.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add a soft AGC in the Q9.23 mel-log domain that automatically clamps
peak energy to prevent clipping on loud utterances, while allowing
recovery toward a 0 dB target at 0.5 dB/sec during active speech.
Requantization is mapped symmetrically from [-1.0, +1.0] in Q9.23 to
[-128, 127] in int8 to match model calibration.

In addition, track a 49-hop VAD history bitmask from the MFCC header.
When the entire sliding context window contains only silence (VAD=0),
int8 requantization and neural network inference execution are gated
off, substantially reducing DSP MCPS load during idle periods.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add playback_compatible_d0i3 and capture_compatible_d0i3 attribute
definitions to the host copier widget class. This allows topologies to
specify D0i3 stream compatibility flags directly on host copier widgets,
enabling the DSP to enter and remain in D0i3 during S0ix while
the stream is active.

Enable capture_compatible_d0i3 on the WoV capture and TFLM detect
host copier widgets in the HDA microphone TFLM Wake-on-Voice
topology.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add Kconfig option COMP_TENSORFLOW_MODEL_FROM_CONTROL to allow
loading the TFLM wake-word model flatbuffer at prepare time from a
configuration data blob via binary kcontrol (bytes control) instead
of using the static C array compiled into the firmware binary.

When this option is enabled, the firmware omits linking
sof_tflm_quantized_model_data.cc and retrieves the model from the
registered blob handler.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Update sof_tflm_train.py to package the trained tflite model behind
the SOF IPC4 ABI header (struct sof_abi_hdr) and export it to:

  - tools/topology/topology2/include/components/tflm/*.conf for
    embedding into ALSA topology v2 files.
  - tools/ctl/ipc4/tflm/*.txt for runtime application with sof-ctl.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Enable CONFIG_COMP_TENSORFLOW_MODEL_FROM_CONTROL in the cAVS 2.5 board
configuration so that the TFLM wake-word model is loaded from runtime
bytes control instead of relying on the built-in static C array.

Add bytes kcontrol with max 65536 to the tflmcly widget in the HDA
Wake-on-Voice detection pipeline topology so that the model flatbuffer
can be set at runtime via sof-ctl.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>